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ADDRESSING THE ALIGNMENT CHALLENGE 
ASSOCIATED WITH THE USE OF COLLEGE 
ADMISSIONS TESTS UNDER ESSA 


EXECUTIVE SUMMARY 


State assessment leaders are concerned about the challenges with receiving full approval from the 
U.S. Department of Education (USED) when using the ACT or the SAT as their assessment of high 
school achievement. USED has asked states opting to use the ACT or the SAT as the achievement 
indicator under the Every Student Succeeds Act (ESSA) to address gaps in alignment between their 
chosen test and their state academic content standards. However, under the current peer review 
criteria, it does not appear that those gaps can be addressed without considerable modification to 
or augmentation of the ACT and SAT and/or revision of state academic content standards. 
Assessment leaders justifiably feel that these two courses of action defeat the purpose of ESSA’s 
flexibility to use these tests. 


In this paper, | make the case that it is necessary to move beyond current peer review requirements 
to resolve this issue. After summarizing the ESSA provision for using locally selected, nationally 
recognized high school academic assessments, | articulate the policy problem concerning this provision: 


The flexibility intended by ESSA’s exception allowing a state to permit districts to use nationally recognized 
high school academic assessments, has at the same time rendered it difficult or impossible for that state 
to adopt a nationally recognized high school academic assessment as its chosen statewide test. 


This problem has led to the current unsustainable process for meeting federal requirements. | 
propose that peer review guidelines be amended to constrain the criteria for judging the degree to 
which a nationally recognized high school academic assessment meets the federal requirements for 
use as an achievement indicator. Simply put, such a test should meet all requirements as fully as it 
can without requiring a substantial modification. This can be resolved with a special peer review 
focused specifically on alignment. If there is disagreement between peer reviewers and states/ 
vendors on how large a change is warranted, the USED can arbitrate. 


INTRODUCTION Assessment leaders feel that 
State leaders are concerned about the challenges with 
receiving full approval from the U.S. Department of securing full approval under 
Education (USED) when using the ACT or the SAT as their 
assessment of high school achievement. Through the peer 
review process, USED has asked states opting to use the to an impasse. 

ACT or the SAT as the achievement indicator under the 

Every Student Succeeds Act (ESSA) to address gaps in alignment between their chosen test and their 
state academic content standards. Alignment studies have revealed problems increasingly pointing 
to test modification and/or revision of state academic content standards as the only way forward. 
Assessment leaders feel that the options open to them for securing full approval under the peer 
review process has led to an impasse. 
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the options open to them for 


the peer review process has led 


In this paper, | make the case that it is necessary to move beyond current peer review requirements 
to resolve this issue. After summarizing the ESSA provision for using locally selected, nationally 
recognized high school academic assessments, | articulate a policy problem concerning this 
provision, which has led to the current unsustainable process for meeting federal requirements. | 
conclude with a proposed solution that acknowledges the complexity of the problem and advocates 
a criterion and process for meeting technical/peer review requirements for states wishing to adopt 
the ACT, SAT or a similar test as an ESSA achievement indicator. 


THE ESSA PROVISION FOR THE USE OF LOCALLY SELECTED, 
NATIONALLY RECOGNIZED HIGH SCHOOL ACADEMIC ASSESSMENTS 


The Every Student Succeeds Act (ESSA, 2015) made changes to the academic assessment provisions 
of part A of title | of the Elementary and Secondary Education Act of 1965 (ESEA). (Title | Federal 
Register, 2016) Among these was the addition of an exception whereby a state may allow its LEAs to 
administer a locally selected, nationally recognized high school academic assessment in lieu of the 
statewide high school assessment. (Title | Federal Register, 2016) 


Under ESSA, an LEA may administer a locally selected assessment “in lieu of the State-designed 
academic assessment” if the LEA selects “a nationally recognized high school academic assessment.” 
(20 U.S.C. 8 1111(b)(2)(H)(i)) The law requires a state to establish technical criteria if it chooses to 
make such assessments available for selection by LEAs, and calls for a review process, led by the 
SEA, to determine if the chosen assessment meets these technical criteria. (20 U.S.C. 8 1111(b)(2)(H) 
(ii)) The law requires the SEA to submit evidence through a peer review process, established by ESSA 
under subsection 1111(a)(4) of the law (20 U.S.C. 8 1111(b)(H)(iii)(II)), after which the SEA may 
approve the assessments for use by any LEA in the state. (20 U.S.C. 8 1111(b)(2)(H)ciii)(II)) 


ESSA specifies the four requirements which must be addressed by a State’s technical criteria for 
approval of a locally selected assessment. (20 U.S.C. 8 1111(b)(2)(H)(v)) These requirements are that 
the assessment must: 


1. “be aligned to the State’s academic content standards” (20 U.S.C. 8 1111(b)(2)(H)(v)(1)); 


2. “provide comparable, valid, and reliable data on academic achievement, as compared to 
the State-designed assessments, for all students and for each subgroup of students” (20 
U.S.C. 8 1111(b)(2)(H)(v)(IN)); 


3. meet all the requirements that statewide assessments must meet under the law, except 
for the requirement that assessments be administered to all students; and 


4. “provide unbiased, rational, and consistent differentiation between schools within the 
State to meet the requirements” outlined in the statewide accountability system section of 
ESSA (20 U.S.C. 8 1111(b)(2)(H)(v)(IV)). 


These requirements are codified in the Code of Federal Regulations (C.F.R.), which also defines a 
nationally recognized high school academic assessment as “an assessment of high school students’ 
knowledge and skills that is administered in multiple States and is recognized by institutions of 
higher education in those or other States for the purposes of entrance or placement into courses in 
postsecondary education or training programs.” (ESSA C.F.R. Section 200.3, 2016) 
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The December 8, 2016 issue of the Federal Register summarizes the changes to title I, part A of the 
ESEA and includes USED’s responses to public comments on the proposed changes. Among the 
responses to comments concerning C.F.R. Section 200.3 (which addresses locally selected, nationally 
recognized high school academic assessments) are those which address state authority over such 
assessments, technical requirements, comparability, and peer review. Four points are worth 
highlighting in the government's responses on these topics. 


First, states clearly have authority to grant and revoke 

LEA use of nationally recognized high school academic First, states clearly have authority 
assessments. A state wishing to implement this authority 
must establish technical criteria but has ultimate authority 
to revoke approval in general for non-technical reasons, nationally recognized high school 
such as a change in state policy. The Federal Register 
discussion on this topic (Title | Federal Register, 2016, p. 
88897) implies that SEAs should prepare LEAs for these 
potential changes, as it envisions that the choice of and implementation of these assessments is 
primarily at the LEA level. And, although the discussion does not preclude it, the underlying 
assumption is that states are not adopting, statewide, a single nationally recognized high school 
academic assessment. Rather, they are (possibly) allowing (some) LEAs to do this. This points to two 
distinct use cases, as noted in Camara, Mattern, Croft, Vispoel, & Nichols (2019a, p.13). The use case 
anticipated by the law is a state granting an LEA permission to implement a nationally recognized 
high school academic assessment. The one not explicitly anticipated by the law is a state adopting 
such a test as its sole statewide assessment. Both use cases are subject to technical requirements 
and peer review, and the policy problem discussed here applies to them equally. 


to grant and revoke LEA use of 


academic assessments. 


Second, with respect to technical requirements, USED responded to comments expressing opposing 
viewpoints on the issue of alignment (to state content standards) for nationally recognized high 
school academic assessments. One viewpoint is that regulations should be revised to clarify that a 
state may only approve assessments that “[measure] the full range of student academic 
performance against the challenging State academic standards.” (p. 88899) The other viewpoint is 
that “the regulations as proposed would preclude the use of one or more assessments.” (p. 88899) 
In response to these concerns, USED indicated that it believed the law already addressed the first 
issue (reifying that such assessments must indeed align fully to a state’s academic content 
standards), and that it understood that some specific assessments might not meet the alignment or 
other technical requirements but that the flexibility to use these assessments is “only appropriate” 
for assessments that “meet all requirements for statewide assessments in general.” (p. 88899) 


Third, the Federal Register discussion concerning comparability focuses on the interpretation of 
comparability and how it is to be assessed; it presumes there is a statewide assessment in place. 
The question of what to make of the comparability requirement in the absence of a distinct 
statewide assessment is not raised. As will soon be noted, comparability is irrelevant for states 
wishing to adopt a nationally recognized high school academic assessment statewide. 


Fourth, USED asserted in its Federal Register responses the appropriateness of peer review for any 
assessment being used for ESSA, including any nationally recognized high school academic 
assessment that a state wishes to implement. Such requirements are indeed outlined in the 
Department's peer review guidance (U.S. Department of Education, 2018), in addition to the 
requirement of comparability to the statewide assessment. 
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| raise these four points to provide context for a potential policy problem for states exercising the 
Section 200.3 option - that is, the option to use, either locally or statewide, a nationally recognized 
high school academic assessment. The policy problem is not about states’ application of the 
Section 200.3 option at the statewide level (the second above-mentioned use case). If a state has 
general authority over (a) whether any district in the state can adopt a nationally recognized test and 
(b) what statewide test the state adopts, then it should have the authority to adopt a nationally 
recognized test as the statewide test (subject to technical requirements and peer review). In this 
scenario, the state has effectively nullified the requirement for comparability to the statewide test. 
We are left with the technical requirements, as vetted through the peer review process, to apply 
to this test. 


THE POLICY PROBLEM CONCERNING NATIONALLY 
RECOGNIZED HIGH SCHOOL ACADEMIC ASSESSMENTS 


When ESSA was enacted into law and during discussions leading up to its codification, support for 
nationally recognized high school academic assessments was premised on the assumption that 
specific known assessments would be eligible candidates. The Code of Federal Regulations (C.F.R.), 
in its response to comments on a different section of the law, Section 200.5 (on assessment 
administration), explicitly mentions the ACT, the SAT, and Advanced Placement tests as tests that 
“would meet the definition of Section 200.3(d)” (Title | 
Federal Register, 2016, p. 88903). In other words, these 
specific tests (though not necessarily only these tests) were 
understood to be “nationally recognized high school and during discussions leading 
academic assessments” during the law-making process. 
Thus, although the law never states it explicitly, the ACT 


and SAT are the key intended tests. nationally recognized high school 


When ESSA was enacted into law 


up to its codification, support for 


There is evidence beyond the Federal Register that academic assessments was 
Section 200.3 was interpreted to refer specifically to the 
ACT and SAT (but not necessarily exclusively to those 
tests). A November 23, 2015 EdWeek summary of the law specific known assessments 
states that “[ESSA] allows for the use of local, nationally- 
recognized tests at the high school level, with state 
permission. So a district could, in theory, use the SAT or 
ACT as its high school test, instead of the traditional state exam.” (Klein, 2015) In January of the 
following year, an EdWeek article claims that “[sS]ources familiar with the discussions that produced 
ESSA said that legislators had the SAT and ACT in mind”, adding that other tests could be considered 
besides these. (Gewertz, 2016) 


premised on the assumption that 


would be eligible candidates. 


The law's writers did not appear to anticipate the possibility that these specific intended tests may 
not meet the same technical requirements as state tests, nor that changes to meet those technical 
requirements might defeat the very purpose of adopting them. 


Not surprisingly, alignment of nationally recognized high school tests to a state’s content standards 
has been the major sticking point in recent discussions on this topic. A special issue of Educational 
Measurement: Issues and Practice offers a sampling of these discussions (see Camara et al., 2019a; 
Camara, Mattern, Croft, Vispoel, & Nichols, 2019b; Dixon-Roman, 2019; Marion & Domaleski, 2019; 
NCME, 2019; Walker & Perie, 2019). 
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It would have been against the law’s intent if the exception 

allowing states to approve and administer these tests in It would have been against the 
lieu of a state-developed test would, when all is said and 
done, merely prove that these tests do not qualify for such 
use. Moreover, unlike state-developed tests, these tests allowing States to approve and 
cannot simply be revised to fit a certain state’s content 
standards. They must meet alignment requirements 
without substantial modifications to content. For one, if tests would, when all is said and 
their content is substantially modified for a given state, 
can they still meet the definition of Section 200.3(d), 
specifically that they are administered in multiple states? tests do not qualify Toe such Use: 
More importantly, modifications to content are 

modifications to an essential feature of a test. A test's 

status as a “nationally recognized” test, with all the validation evidence relevant to its being a desired 
option for a state (Summarized in Camara et al. 2019a), would be to some extent compromised if 
substantial modifications were made, more so if those modifications were state-specific. 


law's intent if the exception 
administer college admissions 


done, merely prove that these 


What about the other side of the alignment equation, state academic content standards? Since 
states have control over their own academic content standards, is it not reasonable to expect a 
state wishing to adopt nationally recognized tests to have, in the first place, academic content 
standards that reflect the content standards of its target nationally recognized test(s)? Perhaps, 
before content standards are adopted by a state, it could have taken into consideration that it may 
one day wish to adopt a nationally recognized high school test - that is, one recognized for college 
admissions or placement. However, given the typical adoption sequence of standards and 
assessments, this is an unlikely scenario. 


Around the time of ESSA, most states were implementing the Common Core State Standards (CCSS) 
as their state academic content standards. These standards highlighted the importance of “college 
and career readiness,” especially at the high school level. ACT and the College Board played a 
prominent role in establishing the CCSS college and career readiness framework. The CCSS was a 
prominent feature of the national standards and assessment landscape at the time the law was 
being written. Legislators would have expected states implementing the CCSS and wishing to adopt 
the ACT or SAT to be able to do so after meeting some technical requirements that can in principle be 
met by these specific tests. Otherwise, ESSA would have been (perhaps unintentionally) setting up 
these states for failure in their attempts to adopt those tests. 


Of course, states can change the standards they have already adopted, but academic content 
standards are hard to change without compelling reasons that go beyond wanting to adopt a 
specific test, and amendment often lies beyond the authority of the SEA (e.g., state boards of 
education, state legislatures). This is especially true in politicized settings where specific content 
frameworks have become contentious. And, as Dixon-Roman (2019) rightly notes “tests should be 
aligned to state curriculums and standards, not states aligning their curriculums and standards to 
the tests. The former ensures that curriculums and defined standards are meeting necessary 
pedagogical and democratic ideals of education, while the latter places substantial power in the 
hands of the test developers to influence education and the polity.” (p. 32) 


USED has asked several states using the SAT and ACT for additional evidence of alignment to state 
content standards (Camara et al., 2019a). Some of these have adopted the Common Core State 
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Standards or similar-enough variants of CCSS - the very 

standards that were in play at the time that ESSA (and the The flexibility intended by ESSA’s 
Section 200.3 option) was written. Thus, there is a real F : 
policy problem which can be stated as follows: exception allowing a state to 

The flexibility intended by ESSA’s exception allowing a state to Peel districts to use nationally 
permit districts to use nationally recognized high school recognized high school academic 
academic assessments, has at the same time rendered it 
difficult or impossible for that state to adopt a nationally 
recognized high school academic assessment as its chosen time rendered it difficult or 


statewide test. 


assessments, has at the same 


impossible for that state to adopt 
a nationally recognized high 


POTENTIAL SOLUTION school academic assessment as 
TO THE POLICY PROBLEM its chosen statewide test. 


States, in conjunction with USED, can respond to the policy 

problem in several ways. Maintaining the status quo, in 

which states wishing to use nationally recognized high school academic assessments continue to 
commission alignment studies, resubmitting evidence in hopes of passing peer review, has led to an 
impasse and is unsustainable. 


The solution advocated here is to acknowledge the policy problem and work toward a resolution 
involving accommodation on both the federal and the state/vendor side. It would not require a 
change in ESSA, and only a minor change to peer review guidance. 


Accommodation would mean establishing and agreeing to 


le such as the following: ; 
a nese ee ne The solution advocated here is to 


The criterion for meeting any technical/peer review 
requirement applying to the use of a nationally recognized 
high school academic assessment cannot be such that that and work toward a resolution 
test would not be able to meet that requirement without involving accommodation on 


substantial modification to the test itself. 
both the federal and the state/ 


acknowledge the policy problem 


What is “substantial modification”? | propose that it is a ; 

change large enough that persons with the appropriate vendor side. 

expertise (such as peer reviewers) would consider the 

modification to significantly affect the applicability of a test’s validation evidence relevant to its 
normal use “for the purposes of entrance or placement into courses in postsecondary education or 
training programs.” The latter quote is from the C.F.R. definition (in Section 200.3(d)) of “nationally 
recognized high school academic assessment.” 


In other words, if peer reviewers believe, for example, that expanding the SAT mathematics subtest 
to include five items assessing a particular geometry standard (and no other changes) would 
significantly alter the SAT’s underlying definition of college readiness, its predictability of first year 
college GPA, and similar criterion-related validation evidence, then that specific change is too great 
an ask for meeting technical/peer review requirements. The standard for meeting these 
requirements must imply a lesser change. Would it be four items? Three? In the extreme, and 
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depending on expert opinion, one might conclude it could 
mean that the standard for meeting the requirements is 
no modification/augmentation at all. However, this is an 
unreasonable position because both the ACT and the SAT 
have themselves undergone systematic content changes 
in their own histories without compromising the applicability 
of earlier validation evidence supporting their use. 


Continuing with this proposed solution, states wishing to 
implement nationally recognized high school academic 
assessments, and their vendors, would be tasked with 
determining the greatest change (including augmentation- 
style changes) that they can make to the test to yield the 
highest alignment gain without compromising the 
applicability of other validation evidence. 


The criterion for meeting any 
technical/peer review 
requirement applying to the use 
of a nationally recognized high 
school academic assessment 
cannot be such that that test 
would not be able to meet that 
requirement without substantial 
modification to the test itself. 


Special peer reviews could be conducted focused on this question exclusively to avoid inefficiencies 
and inconsistencies, with peers either agreeing or disagreeing with state proposed modifications/ 
augmentations. If peers feel that a state has made a good faith effort to address the alignment 
challenge, but also feel that a greater modification/augmentation is warranted consistent with the 
rule above, then they should indicate what that change would be and ideally, present a brief case 


for it. The USED would arbitrate. 


This proposed solution acknowledges that if the criterion for alignment is such that the ACT or SAT 
cannot meet them without becoming different tests, then that criterion runs counter to the 
legislative intent of Section 200.3 and cannot be applied. It acknowledges that it is appropriate to 
expect these tests, as adapted or augmented for state accountability use as an achievement 
indicator, to align in a meaningful way to a state’s high school academic content standards and as 
required by ESSA. It acknowledges that these tests cannot and should not be expected to align as 
well as (or in as strong a sense as) a state-developed standards-based test. And, it respects a state’s 
policy priorities in exercising the option to adopt a nationally recognized high school academic 


assessment as per Section 200.3. 


Alternative courses of action raise problems of their own. 
For example, states can adjust their academic content 
standards in high school to match those of the ACT, SAT, 
or some other chosen nationally recognized test. As 
mentioned earlier, it is difficult for states to change 
standards, and it is politically untenable (and 
indefensible?) to change standards “so it can match a test 
we want to use.” The likelihood of a state taking this option 
appears low, but it is mentioned here because it is an 
obvious possibility. 


Another possibility: Test vendors might change their tests 


If the criterion for alignment is 
such that the ACT or SAT cannot 
meet them without becoming 
different tests, then that criterion 
runs counter to the legislative 
intent of Section 200.3 and 
cannot be applied. 


(substantially if necessary) to better align with state standards. They might develop state-specific 
“School Day” versions of their tests to help states meet technical/peer review requirements, 
counting on it being acceptable to USED and other stakeholders that they would have effectively 
created customized (and no longer “nationally recognized”) statewide tests in situ through a 
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backdoor process. This course of action seems counter to the spirit of the ESSA Section 200.3 
exception, which is to allow states to use the tests as they are or at least with changes that do not 
alter their essential features. 


State can also challenge ESSA directly. They can argue that the ESSA law is inconsistent with its 
original intent, by appealing to a common sense understanding that the spirit of the law is that the 
state has implicit power to select the ACT, the SAT, or a similar test as its statewide test and that the 
technical requirement of content alignment with existing state standards should be nullified or at 
least heavily down-weighted. They might argue that the appropriate criterion for a state-adopted 
test to be aligned to state standards must always be lower than the criterion for a state-developed 
test, because only in the latter case would the state standards have been used explicitly in test 
development. Challenging ESSA in this way could lead in turn to legislative changes. But why go 
down this contentious road before attempting an accommodation with reasonable constraints? 


Finally, states can reverse course on nationally recognized statewide tests, un-adopting them. It 
would be stunning for a state to do this, because of the politics and the high dollar cost of having to 
then procure and pay for a state-developed test. A decision like this would likely have to be driven 
by a serious “School Day” technology or similar failure and the inevitable contractual dispute over 
penalties. Of course, if the adoption of a nationally recognized test is not driven by a defensible 
theory of action supporting its use as the ESSA high school achievement indicator, then a state’s 
right to use it should not be exercised. 


SUMMARY 


According to ESSA, states wishing to use a nationally recognized test, and their vendors, must 
comply with all technical requirements to allow the test to be adopted. These include all technical 
requirements for approving a state-developed test. This compliance requirement for adopting 
nationally recognized tests has revealed a real policy problem, manifested in the current and 
unsustainable peer review process for ascertaining that technical requirements have been met for 
these tests. The heart of the problem is that demonstrating alignment between the test and state 
standards might imply changes to the test (or state standards) that defeat the legislative intent of 
allowing states to adopt that test in first place. 


In this paper, | propose a solution that is premised on the principle that states should be expected 
to meet all technical/peer review requirements with the exception of any requirement, or any criterion 
for meeting a requirement, that alters the essential features of their chosen test as permitted by ESSA. A 
change in content large enough to allow a test to align to state content standards would likely also 
alter the tested construct and defeat the purpose of opting to use it. These types of changes to 
essential features of nationally recognized high school tests should be excluded from any process to 
approve these tests under the ESSA Section 200.3 exception. But smaller changes are both possible 
and appropriate. | propose both a rule for determining how large a change can be made, anda 
small modification to the peer review process, for moving past the present impasse. 
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